Linear Constrained Rayleigh Quotient Optimization: Theory and Algorithms

We consider the following constrained Rayleigh quotient optimization problem (CRQopt) $$ \min_{x\in \mathbb{R}^n} x^{T}Ax\,\,\mbox{subject to}\,\, x^{T}x=1\,\mbox{and}\,C^{T}x=b, where $A$ is an $n\times n$ real symmetric matrix and $C$ m$ matrix. Usually, $m\ll n$. The also known as eigenvalue in literature because it becomes if linear constraint $C^{T}x=b$ removed. start by equivalently transforming CRQopt into problem, called LGopt, of minimizing Lagrangian multiplier CRQopt, then QEPmin, finding smallest a quadratic problem. Although such equivalences has been discussed literature, appears to be first time that these are rigorously justified. Then we propose numerically solve LGopt QEPmin Krylov subspace projection method via Lanczos process. basic idea, for reduce projecting them onto subspaces yield problems same types but much smaller sizes, reduced some direct methods, which either secular equation solver (in case LGopt) or eigensolver QEPmin). resulting algorithm algorithm. perform convergence analysis proposed obtain error bounds. sharpness bound demonstrated artificial examples, although applications often converges faster than bounds suggest. Finally, apply semi-supervised learning context clustering.